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Abstract 
Students in the elementary grades often experience difficulty setting up and solving word 
problems. Using an equation to represent the structure of the problem serves as an effective tool 
for solving word problems, but students may require specific pre-algebraic reasoning instruction 
about the equal sign as a relational symbol to set up and solve such equations successfully. We 
identified students with mathematics difficulty (n = 138) from a sample of 916 third-grade 
students. We randomly assigned students to a word-problem intervention with a pre-algebraic 
reasoning component, a word-problem intervention without pre-algebraic reasoning, or the 
business-as-usual. Students in the 2 active intervention conditions participated in 45 individual 
sessions and learned about 3 additive word-problem schemas. Students who received word- 
problem intervention with a pre-algebraic reasoning component demonstrated improved 
nonstandard equation solving, equal sign understanding, and word-problem solving compared to 
students in the other two conditions. 
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1. Introduction 

In the United States (U.S.), mathematics competency in the elementary grades is largely 
assessed through student performance on solving word problems. Word problems include a 
combination of words and numbers that require interpretation by the student. Many students, 
however, are inadequately prepared to set up and solve word problems (Garcia et al. 2006). For 
students with mathematics difficulty (MD), word problems prove especially challenging due to 
the multiple steps required to develop a solution. Students with MD also struggle to understand 
the symbols of mathematics, including how to interpret symbols within equations (Powell et al. 
2016). In this study, we implemented two variants of a word-problem intervention for students 
with MD to understand the potential added benefit of pre-algebraic instruction embedded within 
a word-problem intervention. 
1.1 Students with MD 

Students identified with learning disabilities in mathematics, sometimes referred to as 
dyscalculia (e.g., Butterworth 2010), account for approximately 3 to 6% of all school-age 
students (Shalev et al. 2000). Beyond disability, however, many students experience MD without 
a formal disability diagnosis (Sztics and Goswami 2013). In this paper, we use the umbrella term 
MD to describe both students with a specific learning disability in mathematics (e.g., students 
with dyscalculia and/or challenges with mathematics calculations and reasoning) as federally 
outlined in the U.S. Individuals with Disabilities Education Act and students who experience 
persistent low mathematics performance without a disability diagnosis. 

Compared to students without MD, students with MD exhibit lower performance with 
counting, comparison, whole-number computation, rational-number understanding, pre-algebraic 


reasoning, and word problems (De Smedt and Gilmore 2011; Fuchs et al. 2013; O’Shea et al. 
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2016; Tolar et al. 2016) and are at great risk for school failure (Wei et al. 2013). For example, 
70% of children who perform below the 10th percentile in mathematics at the end of 
kindergarten receive a diagnosis of MD by fifth grade (Morgan et al. 2009), and over 95% of 
students with MD in fifth grade continue to demonstrate performance below the 25th percentile 
in high school (Shalev et al. 2005). 

Students with MD require intensive and specialized intervention to address mathematics 
challenges (Gersten 2016; Mononen et al. 2014). Without such support, mathematics 
performance gaps persist or widen across grade levels (Koponen et al. 2018; Stevens et al. 2015). 
Across the elementary grades, interventions focused on early numeracy (Clarke et al. 2019), 
whole numbers (Zhang et al. 2014), fractions (Fuchs et al. 2016), and word-problem solving 
(Swanson et al. 2013) have led to improved mathematics performance for students with MD. In 
all of these interventions, researchers employed instructional design principles of explicit 
instruction (e.g., explicit modeling, practice opportunities, feedback) to teach different 
mathematics concepts and procedures to students with MD (Doabler et al. 2017). 

1.2 Word Problems and Students with MD 

Word problems help students connect mathematics to real-world experiences (Depaepe et 
al. 2010). Although many students describe word problems as difficult (Jitendra et al. 2007), 
students with MD perform below students without MD on tests of word-problem solving (e.g., 
Lai et al. 2015; Peake et al. 2015). Word problems often require students to read a key and 
number a graph, understand the problem situation, build the situation model, determine the 
needed operation(s) for solving the problem, interpret and evaluate the problem, solve the 
problem correctly, and add a label corresponding to the number answer (Verschaffel et al. 2000). 


Without explicit instruction on how to set up and solve word problems, students exhibit 
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frustration as they attempt to solve word problems without any explicit steps. Thus, many 
students attend to superficial cues in the word problem and add or subtract without interpreting 
or considering a mathematical model (Van Dooren et al. 2006; Verschaffel et al. 2007). Students 
with MD may use the word-problem text to understand the problem, identify missing 
information, construct a picture or equation, and derive the calculation for finding the missing 
information (e.g., Fuchs et al. 2008). However, students with MD often select the incorrect 
operation(s) for solving the word problem, misuse irrelevant information (Kingsdorf and Krawec 
2014; Krawitz et al. 2018; Wang et al. 2016) and fail to develop a mental model based on the 
text description (van Lieshout and Xenidou-Dervou 2018). Word problems become increasingly 
challenging for students with MD when multiple steps or operations are required to solve the 
problem (Boonen et al. 2016). 

Over the past two decades, researchers have focused on using schemas to help students 
solve word problems (Cook et al. 2019). In schema instruction, students first recognize word 
problems as belonging to a specific problem type and then apply a strategy to solve the problem 
type (Xin and Zhang 2009). Developing schemas for categorizing word problems proves 
beneficial for helping students identify novel problems as belonging to familiar categories (e.g., 
Cooper and Sweller 1987; Ng and Lee 2009). Recognizing schemas to categorize word problems 
also greatly influences whether students answer problems correctly (Kintsch and Greeno 1985) 
and has been reported as more effective than other techniques for teaching word-problem solving 
to students with MD (Jitendra et al. 2015; Zhang and Zin 2012). 

For addition and subtraction word problems, word problems feature three schemas: Total, 
Difference, and Change (Fuchs et al. 2008; Riley and Greeno 1988). In Total problems, also 


called combine or part-part-whole problems, students combine amounts for a total. Students may 
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solve Total problems where the total is missing or one of the parts is missing. Total problems 
also may include more than two parts. In Difference problems, also called compare problems, 
students compare two amounts for a difference. Students may solve Difference problems where 
the difference is missing, the greater amount is missing, or the lesser amount is missing. In 
Change problems, students start with an amount and the amount increases or decreases to a new 
end amount. Students may solve Change problems where the start amount is missing, the change 
amount is missing, or the end amount is missing. Change problems also can present with more 
than one change. 
1.3 Pre-Algebraic Reasoning 

We define pre-algebraic reasoning using the definition provided by Pillay et al. (1998), 
which involves solving equations with one unknown and understanding the equal sign as 
relational (i.e., each side of the equation has the same value). Students with MD experience 
difficulty solving equations with an unknown (Driver and Powell 2015) demonstrating lower 
performance than students without MD. Solving equations with an unknown is tied to 
interpretation of the equal sign (Powell et al. 2015). Many students with and without MD 
misinterpret the equal sign as operational (e.g., Matthews and Rittle-Johnson 2009; McNeil 
2008; Powell et al. 2015) rather than understanding the equal sign as relational. Students 
frequently misunderstand the equal sign because early elementary school instruction and 
textbooks focuses almost exclusively on presenting equations in standard form (e.g.,2 +3 =_ ; 
Capraro et al. 2007; Powell 2012). 
1.4 Connection Between Pre-Algebraic Reasoning and Word Problems 

Teaching students to rely on an equation to represent a word problem’s structure benefits 


students with MD (e.g., Fuchs et al. 2010). The equation provides a visual reference for the word 
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problem, and visuals often aid students with MD (Swanson et al. 2013). Research evidence has 
emphasized the success of teaching pre-algebraic reasoning (i.e., a focus on relations, operations, 
and manipulation to solve for unknowns; Kieran 2004) to third-grade students with MD. 
However, a need exists for additional instructional components focused on solving math 
equations for students with MD to learn pre-algebraic skills within word-problem intervention. 
In the current study, we utilized an updated and expanded version of a word-problem 
intervention called Pirate Math. High-quality studies at second grade (Fuchs et al. 2014) and 
third grade (Fuchs et al. 2010) demonstrated improved word-problem outcomes for Pirate Math 
students compared to the control group. Like pirates who search for a treasure marked with an 
“X,” students in the Pirate Math program find “X” by solving equations (e.g., 5 + X = 14) to 
represent the structure of the word problem (e.g., Maureen had $5, and then she earned money 
from selling a book. Now, Maureen has $14. How much money did she earn selling the book?). 
Powell and Fuchs (2010) conducted a pilot study in which third-grade students with MD 
received Pirate Math, Pirate Math plus equal-sign instruction, or business-as-usual (BAU). 
Students in the two Pirate Math conditions received individual intervention for five weeks, three 
times a week, for 25 to 30 min per session. For the students in Pirate Math plus equal sign 
instruction, tutors defined the equal sign as meaning the same as and instructed students to 
decide if this side of the equal sign is the same as that side. Students used manipulatives, 
pictures, and equations during equal-sign instruction. Powell and Fuchs (2010) conducted a 
mediation analysis to explore the connection between equal-sign understanding and word- 
problem solving (MacKinnon et al. 2007). Results indicated equal-sign instruction significantly 
affected both equation solving and word-problem solving, but equation solving mediated the 


effects of word-problem intervention; that is, equal-sign instruction significantly impacted 


PRE-ALGEBRAIC REASONING WITHIN INTERVENTION 8 


equation-solving ability, which in turn significantly affected word problem-solving ability. 
1.5 Purpose and Research Questions 

In this study, we expanded the pilot study of Powell and Fuchs (2010) to investigate the 
effects of a pre-algebraic reasoning component on the word-problem performance of students 
with MD. Our expansion included: (1) A focus on Total, Difference, and Change problems 
across 16 weeks; the pilot study only was implemented for 5 weeks with Total problems. (2) A 
focus on Total problems across 13 sessions, instead of 10, with continued practice on Total 
problems across 45 sessions of the intervention. (3) A pre-algebraic reasoning component 
focused on understanding the equal sign and setting up and solving equations; the pilot study 
only focused on understanding the equal sign as relational. (4) Additional practice opportunities 
with a balance scale and manipulatives to understand the equal sign as relational and with 
student-generated drawings to solve equations. (5) Modeling and practice within the pre- 
algebraic reasoning component on identifying the variable, drawing a vertical line down from the 
equal sign to visually separate two sides of an equation, and adding or subtracting a constant 
from both sides of the equation to isolate the variable, all of which are new activities in the 
present study. (6) Larger sample sizes with statistical power to determine differences among 
conditions. 

We asked the following research questions: What is the performance growth among 
students with MD within conditions on measures of standard equation solving, nonstandard 
equation solving, and equal-sign understanding? What is the performance growth among 
students with MD within conditions on double-digit additive word problems? 

2. Method 


2.1 Context and Setting 
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We recruited elementary schools from a large urban school district in the Southwest of 
the U.S. The school district served over 80,000 students. In 2017, the district reported 55.5% of 
students as Hispanic, 29.6% as Caucasian, 7.1% as African American, and 7.7% as belonging to 
another racial or ethnic category. In the district, 27.1% of students qualified as English learners, 
and 12.1% received special education services. Overall, 52.4% of students qualified as 
economically disadvantaged. 

2.2 Participants 

We recruited third-grade teachers for study participation from 13 different elementary 
schools within the district. During the 2016-2017 school year, we worked in 52 classes with 37 
teachers. Several schools used departmentalization (i.e., the same teacher taught multiple 
mathematics classes), which accounted for the different numbers of teachers and classes. From 
these 52 classes, we screened 916 third-grade students. 

We used a measure of Single-Digit Word Problems (Jordan and Hanich 2000) to screen 
for difficulty in the area of the mathematics content of the intervention. For study eligibility, we 
identified students who answered 7 or fewer items correctly (out of 14) as experiencing 
mathematics difficulty (MD). This cut off score of 7 represented performance at or below the 
25th percentile. Based on the initial screening, we identified 236 students with MD. Of the 236, 
we determined 91 students with MD as ineligible, and we did not pretest these students for the 
following reasons: no consent; Limited English Proficiency; disability and receiving other 
services; moved before pretesting finished; behavior issues identified by teacher; too many 
students with MD in one classroom; teacher dropped out of study; parent opted out of study; 
truancy; could not schedule. 


We randomly assigned, blocking by classroom, the 145 remaining students to one of 
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three conditions. The three conditions included: Pirate Math Equation Quest (PMEQ; n = 45), 
Pirate Math without Equation Quest (PM-alone; n = 46), and BAU (n = 54). From the start of 
intervention through posttesting, five PMEQ students and one PM-alone student left the study 
because the student moved schools or the teacher wanted the student removed from the study. At 
posttest, we determined one BAU student moved schools. Therefore, across conditions we noted 
4.8% overall attrition. Within conditions, we noted 11.1% attrition for PMEQ, 2.2% for PM- 
alone, and 1.9% for BAU. 

In all, 138 students with MD completed posttesting. Table 1 presents the demographic 
information for the 138 students with complete data. At pretest, we calculated the average age of 


students as follows: 8 years, 8 months for PMEQ; 8 years, 9 months for PM-alone; and 8 years, 8 


months for BAU. 
Table 1 
Demographic Information by Tutoring Condition 
PMEQ PM-alone BAU 
(n = 40) (n= 45) (n = 53) 
Variable n (%) n (%) n (%) 
Female 24 60.0 22 48.9 32 60.4 
Race 
African American 3 13 6 13.3 4 7.5 
Asian American 1 2.5 1 2:2 1 1.9 
Caucasian 3 La 3 6.7 5 9.4 
Hispanic 30 75.0 35 77.8 39 73.6 
Multi-racial 2 5.0 0 0.0 2 3.8 
Other 1 2.5 0 0.0 2 3.8 
School-identified disability 2 5.0 7 15.6 7 13.2 
English learner 29 72.5 27 60.0 30 56.6 
Retained 2 5.0 6 13.3 2 3.8 


Note. PMEQ = Pirate Math Equation Quest; PM-alone = Pirate Math without Equation Quest; BAU = 
business-as-usual comparison. 


2.3 Tutors 


We recruited 15 tutors to conduct the pretesting, intervention, and posttesting. All tutors 
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were pursuing or had obtained a Master’s or doctoral degree in an education-related field. Of the 
15 tutors, 86.7% were female (n = 13) and 13.3% were male (n = 2). Of the tutors, 53.3% 
identified as Caucasian (n = 8), 26.7% as Hispanic (n = 4), 13.3% as Asian American (n = 2), 
and 6.7% as African American (7 = 1). Throughout the year, tutors participated in trainings to 
ensure strong preparation for all aspects of the intervention. In late August and early September, 
tutors participated in three, 3-hr pretesting trainings. In early October, the team participated in 
two, 1.5-hr trainings about the content of the intervention and Total problems. Two subsequent 
1.5-hr trainings followed in November to introduce Difference problems and in January to 
introduce Change problems. Tutors participated in one, 1.5-hr posttesting training meeting. 

2.4 Intervention 

Tutors conducted sessions three times per week (i.e., 45 completed sessions) for 30 min a 
session. We aimed for students to complete 48 of 51 intervention sessions, but after accounting 
for student absences and schoolwide testing, we considered students who completed at least 45 
sessions as completing the entire intervention. The tutors worked with students in a quiet place 
outside of the classroom. We assigned tutors to students in both PMEQ and PM-alone groups to 
ensure an even quality of tutors across the two intervention conditions. 

PMEQ and PM-alone students participated in five activities for each session. Because the 
word-problem intervention is called Pirate Math, all activities used pirate-themed naming. For 
example, pirates are buccaneers (Buccaneer Problems) who embark on quests for treasures 
(Equation Quest). Pirates keep their pirate ships in shipshape (Shipshape Sorting) and fly the 
jolly roger on the ship’s mast (Jolly Roger Review). We named on activity Pirate Crunch 
because students worked quickly through the activity (“in a crunch’). Figure 1 provides samples 


of materials. 
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Figure 1. Sample materials from intervention. 
2.4.1 Math fact flashcards. To increase math fact fluency, tutors displayed a set of math 
fact flashcards (addends 0 to 9; minuends 0 to 18; and subtrahends 0 to 9). After setting the 


timer, students answered as many flashcards as they could in 1 min. The tutors placed cards with 
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a correct response on the desk and provided immediate, corrective feedback for incorrectly 
answered cards. After 1 min, tutors and students counted the number of flashcards answered 
correctly. Prior to starting a second 1-min timing, the tutors challenged students to beat their 
previous score. At the end of the second 1-min timing, students graphed the highest score from 
the two trials. 

2.4.2 Equation Quest. For PMEQ students only, Equation Quest served as the second 
activity of each intervention session. For approximately 2 to 5 min each session, tutors provided 
pre-algebraic reasoning instruction on solving equations and the meaning of the equal sign. 
Tutors reintroduced the common symbol and taught students to understand the meaning of the 
equal sign as the same as. Students learned the equal sign acts as a balance between two sides of 
an equation and does not solely signal a calculation. To understand the equal sign as a relational 
symbol, students solved standard and nonstandard equations with concrete manipulatives (e.g., 
balance scale and blocks), hand-drawn pictures, or equations presented with numbers and 
symbols. Students learned a set of steps to balance equations with a variable (i.e., “X’’), which 
involved isolating the variable and emphasizing that the calculation performed on one side of the 
equal sign also is performed on the other side of the equal sign (e.g., subtract 4 from both sides). 
Students practiced isolating the variable with both standard and nonstandard equations. For all 
PMEQ students, tutors emphasized the meaning of the equal sign as the same as and embedded 
equation solving throughout each session. 

2.4.3 Pirate Crunch. For PM-alone students only, Pirate Crunch served as the second 
activity of each intervention session. For approximately 2 to 5 min each session, students 
completed a mathematical review activity. Pirate Crunch addressed concepts of telling time, 


money, geometry, perimeter, area, place value, and fractions through paper-and-pencil tasks. 
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2.4.4 Buccaneer Problems. The third activity for each session consisted of tutor-led 
schema instruction through a series of three Buccaneer Problems. Note that PMEQ and PM- 
alone students received identical Buccaneer Problems. During sessions | through 4, tutors 
reviewed addition and subtraction skills. Starting in session 5, the tutors provided explicit, 
scaffolded instruction on how to set up and solve word problems by schema. Students learned to 
approach any word problem by RUNning through the problem: Read the problem, Underline the 
label and cross out irrelevant information, and Name the problem type (i.e., choose the correct 
schema to use). For each schema, students learned to use an equation to represent the problem 
and to mark “‘X” to represent the missing information. For the young pirates, “X” represented the 
treasure (i.e., a word-problem answer). The tutors introduced the Total problem schema during 
session 5, the Difference schema in session 17, and the Change schema in session 34. From 
session 39 until the end of intervention, Buccaneer Problems included a comprehensive review 
of Total, Difference, and Change problems. 

2.4.5 Shipshape Sorting. The fourth activity each session, Shipshape Sorting, allowed 
students to practice identifying word-problem schemas learned during the Buccaneer Problems. 
Shipshape Sorting started during session 7 of the intervention. Before the sorting activity began, 
the tutor placed a mat with four squares on the table. Each square was labeled with one word- 
problem type letter (i.e., T for Total, D for Difference, or C for Change) or a question mark. 
Tutors reminded students to sort the word-problem cards and to not solve any of the word 
problems. Tutors set the timer for 1 min and read the first word-problem card aloud before 
handing the card to the students. After 1 min, tutors provided immediate, corrective feedback by 


reviewing at least three of the word-problem cards. 
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2.4.6 Jolly Roger Review. The final activity of each session, the Jolly Roger Review, 
included a brief, timed paper-and-pencil review of the session content. Students worked for | 
min to answer math facts, solve computation problems, or write correct equations for the three 
word-problem schemas. Then, students worked for 2 min to solve a word problem using the 
schema steps taught during the Buccaneer Problems. Students performed the timed review 
independently and then received feedback at the end of the 3 min. 

2.4.7 Motivation. Throughout each session, students earned pirate coins for listening to 
the tutor, staying seated, working hard, and trying their best. Students typically earned 4 to 6 
coins per session. At the end of each session, students counted the number of coins earned and 
colored the appropriate number of coins on a treasure map. When students completed a treasure 
map, they selected a small novelty prize from a treasure box. 

2.5 Fidelity of Implementation 

We collected fidelity of implementation in several ways. First, for pretesting and 
posttesting, the tutors recorded all testing sessions. We randomly selected >20% of audio 
recordings for analysis and measured fidelity to testing procedures against detailed fidelity 
checklists. We measured pretesting fidelity at 98% (SD = 0.003) and posttesting fidelity at 98% 
(SD = 0.049). 

Second, we measured fidelity of implementation of the interventions. We conducted in- 
person fidelity observations once every three weeks for every tutor. We also measured fidelity of 
intervention implementation through analysis of >20% of audio-recorded sessions. Fidelity 
averaged 98% (SD = 0.033) for in-person supervisory observations and 98% (SD = 0.021) for 
audio-recorded intervention sessions. 


Third, all 15 tutors tracked the number of sessions for their PMEQ and PM-alone 
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students. We designed the intervention for students to finish at least 45 sessions with a maximum 
of 51 sessions. The average PMEQ student completed 47.7 days of intervention (range 41 to 50; 
SD = 1.2), and the average PM-alone student completed 47.4 days of intervention (range 38 to 
50; SD = 1.9). 
2.6 Measures 

2.6.1 Pretesting measures. We used Single-Digit Word Problems as the primary 
measure for identifying students with MD (Jordan and Hanich 2000). This measure included 14 
one-step word problems involving sums or minuends of 9 or less categorized into Total, 
Difference, and Change schemas (a = .89). We also administered Texas Word Problems-Brief. 
This measure included eight word problems requiring double-digit computation, with one Total, 
three Difference, and four Change problems, respectively. Cronbach’s a was .85. On Open 
Equations, students solved 10 equations in a standard (e.g.,3 +__ = 8) format. Students also 
solved equations in nonstandard formats, including two identity statements (e.g.,__ = 4), 10 
nonstandard equations with an operator symbol on the right side (e.g.,5 =9-—__), and eight 
nonstandard equations with operator symbols on both sides (e.g.,9 -6 = 7-__). Excluding the 
identity statements, 14 of the equations included addition operator symbols and 14 included 
subtraction operator symbols. Students completed as many problems as possible within the 6 min 
timing. Cronbach’s a was .93. We also administered Equal Sign Tasks (Matthews and Rittle- 
Johnson 2009), which assessed students’ understanding of the equal sign and equivalence in a 
written format. First, the tutors asked students to write a definition of the equal sign. Then, 
students decided if the equal sign was used correctly in nonstandard closed equations. Next, 
students read statements of equivalence and decided whether each statement was always true, 


sometimes true, or never true. Finally, students viewed a closed equation with addends on both 
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sides, divided the equation into two parts, and defined the meaning of the equal sign in the 
equation (a = .78). 

We also administered Texas Word Problems-Part I. Students solved nine double-digit 
word problems: two Total problems, one Difference problem, four Change problems, and two 
multi-schema problems (i.e., Difference and Change; Total and Difference). Two problems 
featured the interpretation of graphs (a = .81). Tutors also administered Equivalence Problems. 
During this 3-min timed activity, students solved 14 open nonstandard equations with operator 
symbols on both sides. All equations included addition operator symbols. Two problems featured 
the same numbers on both side of the equal sign (e.g.,6 +2=6+ _), two problems used the 
same numbers but in reverse (e.g.,4+ | =2 + 4), two problems involved grouping of addends 
(e.g.,2+3+4=2+ _), and the remaining eight problems required solving for a missing part 
without a pattern (e.g.,5 + | =3 +4). Cronbach’s a was .94. Finally, tutors administered Texas 
Word Problems-Part 2. Students solved nine double-digit word problems: two Total problems, 
two Difference problems, three Change problems, one multi-schema problem (i.e., Total and 
Change), and one multiplicative problem (i.e., Equal Groups schema). Three problems featured 
the interpretation of graphs, and one problem included irrelevant information (a = .81). 

2.6.2 Posttesting. During posttesting, tutors administered Open Equations, Texas Word 
Problems-Brief, Texas Word Problems-Part 1, Equal Sign Tasks, Texas Word Problems-Part 2, 
and Equivalence Problems. Tutors followed identical procedures established during pretesting. 

2.6.3 Scoring. Two tutors independently entered scores on 100% of the test protocols for 
each outcome measure on an item-by-item basis into an electronic database, resulting in two 
separate databases. We compared the discrepancies between the two databases across each 


outcome measure and rectified any inconsistencies to reflect the original response. Two tutors 


PRE-ALGEBRAIC REASONING WITHIN INTERVENTION 18 


resolved all discrepancies. Then, we converted students’ responses to correct (1) and incorrect 
(0) scores using spreadsheet commands, which ensured 100% accuracy of scoring. Original 
scoring reliability was 96.4% for pretesting and 99.9% for posttesting. 

2.7 Data Analysis 

For the data analysis, we created several composite scores. We identified standard 
equation solving as the standard equations on Open Equations (maximum score = 10). For the 
students with MD, Cronbach’s o was .77. We calculated nonstandard equation solving as all of 
the nonstandard equations on Open Equations and Equivalence Problems (maximum score = 
34). Cronbach’s o for the nonstandard equation solving composite was .88. Lastly, we created a 
double-digit word problems score by combining Texas Word Problems-Brief, Texas Word 
Problem-Part 1, and Texas Word Problems-Part 2 (maximum score = 52). We calculated 
Cronbach’s o at .92. 

To understand pretest comparability of conditions, we applied ANOVAs to the pretesting 
data using intervention condition (PMEQ vs. PM-alone vs. BAU). We used chi-square analyses 
to determine differences in demographics across the three conditions. To determine if learning 
was a function of intervention condition, we ran ANCOVAS on posttest scores using pretest 
scores as a covariate. We tested a set of orthogonal contrasts using Helmert coefficients (Hinton 
et al. 2004) within a General Linear Model. With the Helmert contrasts, we first determined 
whether one condition (i.e., PMEQ) differed from the two remaining conditions; we then 
assessed whether the remaining conditions (i.e., PM-alone and BAU) differed from one another. 
We calculated effect sizes (ES) by subtracting adjusted posttest means and dividing by the 
unadjusted posttest pooled standard deviation. 


3. Results 
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3.1 Students with MD Compared to Students Without MD 

In this section, we compare the performance of the 236 students initially identified with 
MD to the 680 students without MD. As expected, students with MD (M = 4.69, SD = 2.34) 
performed significantly lower than students without MD (M = 12.17, SD = 1.99) on Single-Digit 
Word Problems, F(1,914) = 2251.63, p < .001. This result represented an ES of 3.44 and 
indicated the students with MD experienced very low single-digit word-problem performance 
compared to grade-level peers. Similarly, students without MD (M = 5.78, SD = 3.74) scored 
higher than students with MD (M = 2.09, SD = 2.15) on Texas Word Problems-Brief, F(1, 914) = 
206.58, p < .001. We calculated an ES of 1.21, which suggested a large gap between the double- 
digit word-problem performance of students with and without MD. 

We noted an analogous pattern of results on the measures related to equation solving and 
the equal sign. On Open Equations, students with MD (M = 5.89, SD = 4.65) demonstrated lower 
average scores than students without MD (M = 13.31, SD = 7.65), F(, 914) = 218.44, p< .001. 
With an ES of 1.17, students with MD demonstrated a substantial deficit in equation-solving 
performance compared to peers without MD. Not surprisingly, students with MD (M = 5.55, SD 
= 3.03) performed lower than students without MD (M = 9.29, SD = 3.68) on Equal Sign Tasks, 
FC, 914) = 197.36, p < .001. On Equal Sign Tasks, we calculated as ES of 1.11. Results from 
each of these four measures indicated our method for identification of students with MD was 
successful. Results also implied this group of students with MD necessitated focused 
mathematics intervention efforts to improve mathematical understanding of word problems, 
equation solving, and the equal sign. 

3.2 Students With MD 


3.2.1 Demographics. Of the 138 students with MD with complete data, we did not 
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identify differences on demographics (sex, race or ethnicity, special education status, English 


learner status, and retained status) as a function of condition. Students also did not differ based 


on age as a function of condition. 


3.2.2 Pretest comparability. Table 2 presents unadjusted pretest and posttest means, as 


well as adjusted posttest means, for students in the three intervention conditions. At pretest, we 


identified no significant differences among intervention conditions on any of the five scores used 


in our analysis: single-digit word problems, double-digit word problems, standard equations, 


nonstandard equations, and equal-sign tasks. 


Table 2 
Pre- and Posttest Performance on Outcome Measures 
PMEQ PM-alone BAU 
(n = 40) (n= 45) (n = 53) 
M____SD*orSE° M SD? or SE° M____SD* or SE® 
Single-Digit Word Problems 
Unadjusted pretest 4.78 1.87 5.20 2.24 3:23 2.02 
Double-digit word problems 
Unadjusted pretest 5.65 3.15 6.89 5.69 7.02 4.84 
Unadjusted posttest 26.12 10.07 23.51 9.55 11.42 5.14 
Adjusted posttest 26.41 131 23.42 1.23 11.28 1.13 
Standard equation solving 
Unadjusted pretest 2.50 2.18 2.51 2.08 2.42 1.97 
Unadjusted posttest 4.90 2.53 5.67 2.39 4.91 2.88 
Adjusted posttest 4.89 0.39 5.65 0.37 4.93 0.34 
Nonstandard equation solving 
Unadjusted pretest 4.85 3.31 4.33 3.08 4.15 4.10 
Unadjusted posttest 10.48 6.89 8.69 5.12 7.42 4.71 
Adjusted posttest 10.17 0.79 8.74 0.74 7.60 0.69 
Equal Sign Tasks 
Unadjusted pretest 6.00 2.51 6.16 3.19 5.51 2.65 
Unadjusted posttest 10.43 2.84 7.24 2.76 7.85 2.78 
Adjusted posttest 10.36 0.39 TAl 0.37 8.01 0.34 


Note. PMEQ = Pirate Math Equation Quest; PM-alone = Pirate Math without Equation Quest; BAU = 


business-as-usual comparison. 


@Standard deviation for unadjusted means. 


Standard error for adjusted means. 


3.3.3 Performance growth. We identified no significant differences on posttest scores of 
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standard equation solving among the three conditions, F(2,134) = 1.36, p = .261. In our 
southwestern state, students typically set up and solve standard equations as part of the third- 
grade mathematics curriculum, so we expected significant improvement for all students. On 
nonstandard equation solving, however, we noted marginally significant differences among 
conditions, F(2, 134) = 3.02, p = .052. Helmert contrasts indicated PMEQ students outperformed 
PM-alone and BAU students (p = .035; ESs = 0.24 and 0.44, respectively). PM-alone and BAU 
students performed comparably (p = .259; ES = 0.23). 

On Equal Sign Tasks, we identified significant differences among conditions, F(2, 134) = 
19.35, p > .001. The Helmert contrasts signaled PMEQ students outperformed PM-alone and 
BAU students (p < .011) with ES of 1.16 and 0.84, respectively. We noted a marginally 
significant (p = .074) difference between PM-alone and BAU students with an ES of 0.33 
favoring BAU students. 

On double-digit word problems, we identified significant differences among conditions, 
F(2, 134) = 45.45, p < .011. Helmert contrasts indicated PMEQ students outperformed PM-alone 
and BAU students (p < .001; ESs = 0.31 and 1.89, respectively), and PM-alone students 
outperformed BAU students (p < .001; ES = 1.58). 

4. Discussion 

With this study, we randomly assigned students with MD to one of three conditions. In 
the two active intervention conditions, we provided a word-problem intervention focused on 
three word-problem schemas. In one of the active word-problem conditions, we examined the 
inclusion of a pre-algebraic reasoning component. We conducted this research to determine if 
instruction about pre-algebraic reasoning, with a focus on the equal sign and solving equations, is 


a necessary component of word-problem intervention. 
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4.1 Students With and Without MD 

Before analyzing pre- to posttest differences of the students with MD, we compared the 
mathematics pretest performance of students with MD versus students without MD. On the 
word-problem measures, we noted significantly lower scores for the students with MD, with ESs 
favoring students without MD ranging from 1.21 to 3.44. This significant difference in word- 
problem performance for students with MD compared to students without MD provided us with 
the rationale to provide targeted word-problem intervention to students with MD. We also noted 
significantly lower performance on two measures of pre-algebraic reasoning, with students 
without MD demonstrating higher scores, with ESs of 1.11 and 1.17. The disparity in pre- 
algebraic reasoning scores of students with MD versus without MD led us to develop and test the 
pre-algebraic reasoning component within the word-problem intervention. 
4.2 Comparisons Across Conditions 

With our first research question, we examined equation-solving performance by 
analyzing performance growth from pre- to posttest with standard equation solving and 
nonstandard equation solving. On standard equations, we detected no significant differences 
among conditions. This finding is not surprising because the majority of equations presented in 
elementary school are standard equations (Powell 2012). Although some standard equations 
require solving for an unknown addend, minuend, or subtrahend (Gilmore, 2006; Hiebert 1982), 
our pre-algebraic reasoning component within PMEQ did not lead to performance differences on 
standard equation solving. 

On nonstandard equations, however, we noted the benefit of the pre-algebraic reasoning 
component within the PMEQ intervention. PMEQ students demonstrated higher posttest scores 


than PM-alone and BAU students with small to medium ESs of 0.24 and 0.44 (Cohen 1992). 


PRE-ALGEBRAIC REASONING WITHIN INTERVENTION 23 


PMEQ students received only 2 to 5 min of practice each session related to the meaning of the 
equal sign and equation solving. Given that solving nonstandard equations requires an 
interpretation of the equal sign as relational (McNeil et al. 2011; Molina et al. 2009), this result 
demonstrates our brief and consistently-implemented Equation Quest activities led to improved 
solving of nonstandard equations. Notably, we observed PM-alone students outperformed BAU 
students on the nonstandard equations (ES = 0.23). We hypothesize the mere action of generating 
equations to represent word problems, which is a common practice in word-problem solving 
(Koedinger and Nathan 2004), by both PMEQ and PM-alone students likely assisted PM-alone 
students with exposure to different equations. The explicit instruction in Equation Quest, 
however, provided a nonstandard equation solving advantage favoring PMEQ students. PMEQ 
students learned how to interpret the equal sign as a balance between two sides of an equation 
and apply this equal-sign interpretation to balancing two sides of an equation during equation 
solving. Students with MD, when provided with explicit modeling and practice opportunities 
related to solving nonstandard equations, can effectively learn to solve difficult equations. 

Equal Sign Tasks assessed students’ understanding of the equal sign through definitions, 
explanations, and equations that did not require solving. As described, students in the PMEQ 
intervention received explicit instruction about a relational interpretation of the equal sign, and 
this instruction led to large and significant gains over PM-alone (ES = 1.16) and BAU (ES = 
0.84) students. This finding shows the value of pre-algebraic reasoning instruction embedded 
within the PMEQ intervention. When students learn an explicit definition of the equal sign as the 
same as and participate in systematic practice opportunities for balancing equations, they learn to 
interpret the equal sign correctly. With brief instruction, equal-sign interpretation shifts from 


operational to relational (Panayides, 2014). With the Equation Quest embedded within PMEQ, 
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students improved their equal-sign understanding substantially, which holds important 
implications for later success with algebra (Pillay et al. 1998; Vincent et al. 2015). 

With our second research question, we investigated the performance growth among 
students on double-digit additive word problems. We calculated large effects for PMEQ students 
over BAU (ES = 1.89) and PM-alone students over BAU (ES = 1.58) on double-digit word 
problems, indicating the word-problem intervention (i.e., Pirate Math) led to enhanced word- 
problem solving for students with MD. When provided with explicit modeling and practice about 
attacking (RUN: Read the problem, Underline the label, Name the problem type), setting up, and 
solving Total, Difference, and Change word problems, these students with MD became more 
efficient problem solvers. This improvement in word-problem performance after participation in 
a word-problem intervention focused on schemas contributes to the research base by supporting 
explicit schema instruction for students with MD (e.g., Flores et al. 2016; Peltier and Vannest 
2017). 

We identified a small effect on double-digit word problems favoring PMEQ students over 
PM-alone students (ES = 0.31). This ES is larger than the ES of 0.22 reported by Powell and 
Fuchs (2010), which highlights the benefit of Equation Quest within the word-problem 
intervention. Our results demonstrate the importance of explicitly teaching students to 
understand the equal sign as a relational symbol and how this interpretation contributes to 
accurate equation solving and effective word-problem solving. 

4.3 Limitations 

We note several limitations to this study. First, to conduct data analyses, such as a 

mediation analysis, we need a larger sample size. When we complete a full efficacy trial in 2019, 


we will have statistical power to run such analyses. At this time, we rely on data from one year 
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of a multi-year study to provide direction for future research. 

Second, once we identified the students with MD, we did not complete any of our 
measures with the students without MD. It would be helpful to understand whether performance 
gains of the PMEQ and PM-alone students facilitated improved learning comparable to students 
without MD as third-grade students maturate in their mathematical learning across the school 
year. As students with MD exhibit below-average performance compared to students without 
MD (Nelson and Powell 2018), researchers should explore whether an effective intervention 
package can alleviate performance gaps between students with and without MD. 

Third, we only assessed students using measures requiring written responses. Students in 
the PMEQ condition participated in a range of hands-on activities related to understanding the 
equal sign and solving equations. Therefore, we may want to assess future cohorts of students 
using tools, such as a balance scale, to determine if written responses accurately capture 
students’ thinking and learning. As demonstrated by Driver and Powell (2015) and Sherman and 
Bisanz (2009), students often exhibit complex mathematical thinking with manipulatives or 
drawn pictures but find the numbers, symbols, and words of mathematics to be stumbling blocks 
hindering successful mathematics performance. 

Fourth, our intervention was administered in a highly controlled setting, with individual 
sessions dedicated solely to word-problem solving occurring across 16 weeks. The nature of our 
intervention may be difficult to replicate in school settings where time, scheduling, and access to 
tutors proves challenging. During the 2018-2019 school year, our research team spearheaded an 
effort to implement small group Pirate Math intervention in schools to address this concern; 
results are forthcoming. 


4.4 Implications for Practice 
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Because students with MD benefit from explicit schema instruction on reading, 
interpreting, setting up, and solving word problems, we recommend teaching students to use an 
attack strategy to solve word problems. Students should use the same general attack strategy to 
solve all word problems. For example, in our intervention, we instructed students to RUN 
through the word problem (1.e., Read the word problem, Underline the label and cross out 
irrelevant information, and Name the problem or schema type). In addition to learning an attack 
strategy, students should learn how to identify and solve the three additive schemas of Total, 
Difference, and Change. As students with MD begin to understand the additive schemas, they 
also need to learn how to write equations to represent a word problem’s structure. Our word- 
problem intervention used explicit tutor modeling and included guided and independent practice 
opportunities for students to become proficient with solving word problems. We suggest teachers 
of students with MD offer similar modeling and practice opportunities. 

We also suggest teachers employ variations of our intervention components to support 
students’ word-problem proficiency. For example, 1 to 2 min of daily addition and subtraction 
fluency practice (e.g., Math Fact Flashcards) should be embedded into mathematics instruction 
to ensure ease of solving equations. Students with MD also may benefit from explicit modeling 
and practice of pre-algebraic reasoning skills (e.g., Equation Quest) such as interpreting the equal 
sign as relational and using equal-sign knowledge to solve different types of equations. We 
recommend teachers of students with MD include multiple representations within their modeling 
and practice opportunities to encourage students to explore the equal sign as a balance. We also 
propose teachers expose students to both standard and nonstandard equations, especially because 
the latter encourage students to think about the equal sign as a relational symbol. Teachers 


should explicitly connect pre-algebraic reasoning with the writing and solving of equations to 
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ensure students transfer pre-algebraic reasoning knowledge to word-problem solving. 

Teachers also should consider activities that build schema fluency (e.g., Shipshape 
Sorting) to support students’ understanding of the different schema types. Lastly, teachers need 
to include independent practice opportunities (e.g., Jolly Roger Review) that assess fluency, pre- 
algebraic reasoning, equation solving, and word-problem solving in their instruction. 
Independent practice opportunities provide an important way to determine the specific word- 
problem challenges for students with MD, which then inform future instructional decisions. 

4.5 Conclusion 

Our results on nonstandard equation solving and equal-sign understanding indicate 
explicit modeling and practice, provided within the Equation Quest component of PMEQ, lead to 
improved scores. In turn, advanced equal-sign knowledge and equation-solving skills suggest 
enhanced performance with solving word problems. Our results demonstrate preliminary benefits 
for the Equation Quest component embedded within an efficacious word-problem intervention. 
As U.S. standards expect students to set up and solve equations with unknowns in all positions 
(e.g., _ —6=10) and interpret the equal sign as relational, a significant need exists to ensure 
students with MD receive intervention focused on improving both of these skills. Additionally, 
as high-stakes assessments require students to set up and solve word problems, students with MD 
need to participate in intervention efforts that improve their conceptual understanding of word 
problems with procedural methods for solving such problems. With our PMEQ condition, we 
have helped students with MD participate successfully in the elementary grade curriculum, and 
we will continue to investigate the added benefit of pre-algebraic reasoning within word-problem 


solving for students with MD. 
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